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ABSTRACT 

An amendment to the Job Training Partnership Act 
(JTPA) mandated an assessment of the basic skills of all Summer Youth 
Employment and Training (SYET) Program enrollees. Many agencies in 
California used the Employability Competency System Appraisal Test to 
assess 1987 SYET participants* basic reading and math skills. Between 
June and October 1987, the test was administered to 10,003 SYET 
enrollees in California. Slightly less than half of those taking the 
test were female, with Hispanics accounting for 43.9 percent, Blacks 
for 22.9 percent, Asians for 15.7 percent, and Whites for 10.8 
percent. English was the native language of 59.6 percent of those 
tested, with 22.6 percent of the youths listing Spanish as their 
native language. Thirty-one percent had completed high school, and 
49.9 percent had completed the 9th or 10th grade. Nearly 11 percent 
of the SYET enrollees needed both math and reading remediation, 19.4 
percent needed only math remediation, 2.9 percent needed only reading 
remediation. It was determined that an additional 32.4 percent would 
benefit from some remediation (that is, they functioned at or above a 
seventh-grade level but below a high school entry level), leaving 
34.4 percent who functioned at or above the high school entry level. 
(Eighteen figures are provided.) Appendixes include a description of 
the components of the Employability Competency System and lists of 
counties, agencies, and/or Service Delivery Areas ordering and using 
the test in 1987. (MN) 
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SUMMER YOUTH EMPLOYMENT 
AND TRAINING PROGRAM 
STATEWIDE REPORT 
ON THE APPRAISAL OF BASIC SKILLS 



Prior to 1987, Summer Youth Employment and Training Programs (SYET) were 
responsible for finding \obs for young people. In-school and out-of-school youth 
between the ages of 14 and 21 who were economically disadvantaged were eligible to 
participate in the program. In February 1987, agencies working with Job Training 
Partnership Act (JTPA) contracts were advised that Public Law (PL) 99-496 amended 
portions of the Job Training Partnership Act (JTPA) including provisions related to the 
Title ll-B Summer Youth Employment and Training (SYET) Program. This amendment 
required all Service Delivery Areas (SDAs) to assess the basic skills of al! SYET youth 
for the 1987 Summer Program. 

This shift in policy reflects research findings which show that young people, in 
particular economically disadvantaged young people, lose ground academically during 
summer vacation. Another reason this amenJment was passed is the recognition of the 
importance of basic skills in a person's ability to get and keep a job. In order to enhance 
a young person's long term chances for successful employment, the program is being 
asked to intervene and determine which young people would profit from basic skills 
remediation, and provkJe educational opportunities in conjunction with job experiences. 

Because of the new amendment, an integral component of the initial appraisal 
of JTPA eligible SYET participants is an assessment of the participanfs basic reading 
and math skills. On the basis of test results, participants who lack basic literacy or 
mathematic skills can be referred for appropriate remediation. 

During the SYET 1987 Program in California, many agencies used the 
Employability Competency System (ECS) Appraisal Test to meet this new 
requirement. The ECS Appraisal Test was designed to provkJe an assessment of the 
participant's level of skill development in the areas of basic reading comprehension 
and basic math computation in an employability context. Both portions of the Appraisal 
were developed by the Comprehensive Adult Student Assessment System (CASAS) 
through a contract administered by the California State Department of Education. The 
two tests together have been designated as the ECS Appraisal Test. Both were 
developed to be included in the initial appraisal component of the ECS. The ECS 
Appraisal Test is only one component of a complete employability competency system 
that links employment competencies to assessment and instructional materials. For a 
description of the entire system, see Appendix A. 

The following report describes the ECS Appraisal Test. It discusses the 
demographics of the 1987 SYET population including the gender, ethnicity, native 
language, age, and educational characteristics. Participant test performance on the 
Reading and Math Appraisal tests is also discussed. 
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EMPLOYABILITY COMPETENCY SYSTEM (ECS) 
BASIC SKILLS APPRAISAL TEST 



The ECS Reading Test is designed to assess a participant's ability to apply basic 
reading skills in a functional context related to training and employment. It consists of 
twenty multiple choice items. 

The ECS Math Test is designed to assess a participants ability to perform basic math 
computations and to apply basic math skills in a functional context related to training 
and employment. This test also consists of twenty items. 

The ECS Appraisal Test was developed from the CASAS Item Bank. This bank of over 
4,000 items has been under continual development and refinement since 1980. The 
application of Item Response Theory (IRT) to these 4,000 items assigns to each item a 
reliable index of standardized difficulty. The test forms developed from these items 
accurately measure basic skills in a functional contex*. 



PSYCHOMETRIC PROPERTIES 

The psychometilc properties of the ECS Appraisal Test forms are briefly summarized 
below and show ttie instrumentation used in ttie EOS Appraisal Test to be internally 
consistent and accurate with the pyschometric model used. 

Reliability Computation of Kuder-Richardson (KR)-20 indices for ECS Reading and 
Math test items indicaied that in the case of the ECS Reading Test, the KR-20 was .77. 
The corresponding figure for ttie ECS Math Test was .85. 

Item-Total Correlations Point bi-serial correlation coefficients were obtained for 
ttie ECS Reading and Matii Tests. This correlation should generally fall between .40 
and .60 for each of the individual test items. In ttie case of the ECS Reading Test, the 
coefficients ranged from .17 to .54, witti a mean value of .44. Similar coefficients for the 
ECS Math Test ranged from .41 to .59 with an average P-Value of .51. 



SCOPE OF REPORT 

Data for this report was gathered from June 1987 through October 1987 for 10,003 
cases. All data reported was taken from the ECS Appraisal answer sheets which were 
filled out by ttie participants at ttie time the test was administered. The counties and 
programs represented in ttiis data base are listed in Appendix B. All agencies which 
used the ECS Appraisal were encouraged to send back the scannable portion of the 
answer sheet. Appendix C contains a list of all the SDA*s that had agencies wno used 
the ECS Appraisal during ttieir SYET Program. 



DEMOGRAPHIC DATA 



Gender. The gender of the participants included In this sample is approximately 
48.5% Female and 51.5% Male. (See Figure 1) 



PARTICIPANT GENDER 
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MALE 
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FEMALE 



51.5% 



N- 10.003 



Insufficient 
Data « 108 



Figure 1 




Age. Approximately 15% of the participant sample is under the age of 15. More than 
one-half of the participants (65%). ari between the ages of 15 and 17, v/hile 
approximately 19% are t)etween the ages of 18 and 21. (See Figure 2) 




Figure 2 
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Ethnicity. Approximately 10% of the participants in this sample are Caucasian, 44% 
are H'^^^oanic and 23% are Black. These three groups comprise approximately 77% of 
the participant sample. The remaining 23% are di^^tributed among Native American, 
Asian, lndo-Ch;nese, Filipino, Pacific Islander, and Other. (See Figure 3) 
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Native Language. English was identified as the native language by approximately 
60% of the participants and Spanish by 23%. Thirteen percent identified their native 
language as Vietnamese, Laotion or Cambodian. The remaining 4% were Korean. 
Chinese, Japanese, Tagalog. and Other. (See Figure 4) 
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Education. Approximately 16% of the sample have completed seven or eight years of 
education. Fifty percent have a minimum of 9-10 years of education. Thirty-one percent 
reported completing 11 or 12 years of education. The mean or average years of 
education for all participants is approximately 9.9. (See Figure 5) Approximately 
17% ropvirt having a high school dinloma. a General Education Development (GED) 
certifier*'*, or have passed the California High School Proficiency Exam (CHSPE; the 
;epal eq. ./alent of a high school diploma in California). Approximately 81% report not 
navmg a degree. (See Figure 6) 
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Figure 5 



HIGHEST DEGREE EARNED 
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CASAS SCALE SCORE INTERPRETATION 



The test results reported in the following sections are imported on the CASAS scale. 
Functional levels of basic skills have been identified based on five years of statewide 
achievement data for students enrolled in Basic Education programs in California. The 
following descriptions of the levels will assist in the interpretation of the test results. 

E ILOW 200 Participants functioning below 200 have difficulty with basic literacy 
and computational skills necessary to function in employment and in the community. 
These participants have difficulty providing basic personal identification in written form 
(e.g. ]0b applications)! are not able to compute wages and deductions on paychecks, 
and cannot follow simple basic written directions and safety procedures. 

200 - 214 Participants functioning between 200 and 214 have low literacy skills 
and have difficulty pursuing other than entry level programs requiring minimal literacy 
skills. They can fill out simple job application forms and demonstrate basic 
computations only. These participants are functioning below a 7th grade level. 

215 - 224 Participants functioning between 215 and 224 are functioning above a 
basic literacy level, and are able to handle basic literacy tasks and basic computational 
skills in a functional setting related to employment. They have difficulty following more 
complex sets of directions and are functioning below a high school levei. 

225 AND ABOVE Participants functioning at or above 225 can function at a 
high school entry level in basic reading and math and if they do not have a high school 
diploma can profit from instruction at the high school level. They can usually perform 
work that involves following oral and written directions in familiar and some unfamiliar 
situations. Those participants 18 years of age and above can profit from instruction in 
General Educational Development (GED) preparation and, in a short time, have a high 
probability of passing the GED test. 
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TEST SCORE PERFORMANCE DATA 



As discussed earlier, the total sampled for this report was 10,003. On the reading test, 
fifty-seven percent of the sample achieved a scale score of 225 or above, while 
approximately 1 .4% achieved less than a 200 scale score. Almost 86% achieved 
higher than a 215 scale score. The mean or average score was 225.15, with a 
standard deviation of 1 1 .22. Scores were more evenly dispersed for the Math Test 
compared to the Reading Test. Approximately 41% achieved above a 225 scale score, 
29% scored between 215 and 224, 24% scored between 200 and 214, while 6% 
scored less than 200. The average score on the Math test was 223.01, with a standard 
deviation of 14.02. (See Figure 7) 

ECS READING AND MATH TEST SCORE PERFORMANCE 




■ ECS READING 
□ ECS MATH 



N = 10,003 



PERCENT 

Figure 7 

Math and Reading. Figure 8 shows the crosstabulation of reading and math 
scores and identifies the percent of students most in need of remediation and the area 
of remediation needed. For example, of those partidpants who scored 225 or above in 
Math, 84% scored above 225 in Reading. By contrast, for those participants scoring 
225 or above in Reading, only oi)% scored at or above 225 in Math. Of those 
participants who scored above 225 in Math, no one scored less than 200 in Reading, 
1.2% scored between 200 and 214, 15% scored between 215 and 224, and 84% 
scored above 225. Of those participants scoring between 215 and 224 on the Math test, 
less than .1% scored less than 200 or the Reading test, 8.4% scored between 200 and 
214, 36.5% scored between 215 and 224, while 55% scared ai or above 225 in 
Reading. 
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SCORE GROUP ESTIMATES BY 
READING AND MATH TEST SCORE 
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ECS MATH SCORES 
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Functioning below a 7th grade level 
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Functioning at or at>ove a High School entry level (34.4%) 



Figure 8 
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Gender. Analyses of test score performance by participant gender are presented in 
Figure 9 for the ECS Reading and Math Tests, respectively. Differences in 
performance t)etween males and females can be noted with respect to scores on the 
Reading Test. A somewhat larger percentage of males scored below a 215 scale 
score (17%) than did females (10%). Sixty-four percent of the females in the sample 
scored above 225 compared to 51% of the males. Little difference existed between 
males and females relative to the percent of participants falling in the various scale 
score categories for Math. 



READING AND MATH TEST SCORtS BY PARTICIPANT GENDER 
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Figure 9 
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Age. Reading and Math achievement levels were positively related to age. Sixty-one 
percent of the eighteen to twenty-one year okJs scored 225 and above in Reading as 
compared to 48 % of the fourteen year olds. (See Hgure 10) i%s noted in Figure 
11 approximately 45% of the eighteen to twenty-one year olds scored 225 and 
above in the Math test as compared to 34% of the under fifteen age group. 
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Figure 10 



MATH SCORE BY PARTICIPANT AGE 
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Figure 11 
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Reading Score by HIghe-' Grade Completed. Participant-reported years of 
education was positively related to achievement levels on the ECS Reading and Math 
tesii. As noted In Rgure 12, among those participants who completed eight or fewer 
years of education, approximately 40% achieved scale scores or 225 and above on 
the Reading Test. The data indicate that participants completing nine to ten years of 
education comprise approximately one-half of the distribution. Among these 
participants, 58% scored 225 or above, and 12% scored below a 215 scale score. 
Among those participants who indicated completion of the 1 1th or 12th grade (31% of 
the distribution), 67% scored above a 225 scale score, while approximately 10% 
scored below 215. 



READING SCORE BY HIGHEST GRADE COMPLETED 




Figure 12 
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Math Score by Highest Grade Completed. Among those participants who 
completed fewer than 9 years of education, only 27% scored above 225, while 43% 
achieved below a 215 scale score. Among those participants completing from nine to 
ten years of education, 43% scored at or above a 225 scale score, while 
approximately 28% achieved below a 215 scale score. As with Reading Test scores, 
completion of the 12th grade had a strong effect on the distribution of scores. Among 
those participants who reported completing the 13th grade, 71% attained a scale 
score of 225 and above, while the percent of participants scoring below 215 dropped 
to approximately 10%. (See Figure 13) 



MATH SCORE BY HIGHEST GRADE COMPLETED 
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Figure 13 
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Reading Score By Highest Degre3 Earned. Among those participants who 
had a high school diploma, 69% scored at 225 or above on the Reading Test, 21% 
scored between 215 and 224, while the remaining 10% scored below 215. Among 
those participants without an educational degree, the percent achieving above a 225 
scale score dropped to 56% on the Reading Test, and the percent of participants 
scoring bolow 215 increased to approximately 13%. (See Rgure 14) 



READING SCORE BY HIGHEST DEGREE EARNED 




Figure 14 



ERIC 



2i 

16 



Math Score By Highest Degree Earned. On the ECS Math Test, among those 
persons with a high school diploma, approximately 53% scored above a 225 scale 
score, 25% scored between 215 and 224, 19% scored between 200 and 214, while 
3% scored less than 200. Among those participants lacking formal high school 
completion, only 40% scored 225 and aoove, and 30% scored t)elow 215. (See 
Figure 15) 



MATH SCORE BY HIGHEST DEGREE EARNED 




Figure 15 
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Ethnicity. Figures 16 and 17 pref^ent ECS Appraisal Reading and Math Test 
scores with participant ethnic background. As indicated in Rgure 16, 58% percent 
of the Caucasians had scores of 225 and above on the ECS Reading Test as 
compared to 61% of the Hispanics, and 56% of the Blacks. Little difference also 
existed between these three groups in the percent of respondents scoring below a 
200 scale score. On the ECS Math Test, 39% of the Caucasians achieved scale 
scores of 225 and above, compared to 44% of the Hispanics and 29% of the Blacks. 
(See Figure 17) Nine percent of the Caucasians achieved below a 200 scale 
score, compared to 4% of the Hispanics, and 8% of the Blacks. 
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Figure 16 



MATH SCORE BY PARTICIPANT ETHNICITY 
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APPENDIX A 



The Employabllity Competency System 

The 1982 Job Training Partnership ACT (JTPA), Section 106, (b) (2) addressed the 
need to include the attainment of pre-employment competencies recognized by the 
local Private Industry Councils (PICs) as one option for youth participants to achieve 
a positive terminaticn from a program. In response to this, PIC's across the nation 
have established a variety of processes to begin identifying relevant employabllity 
competendeSp appropriate curriculum plus valid and reliable assessment. The task 
of developing, implementing and evaluating an employabllity competency system that 
meets the identified standards of effectiveness and relevancy is time consuming and 
requires the expertise of curriculum, training, and assessment specialists. 

The California State Job Training Coordinating Council identified this as a statewide 
priority need in 1985-86 and the California State Department of Education, in 
coordination with the JTPO, contracted with CASAS (Comprehensive Adult Student 
Assessment System) to begin developing a comprehensive Employabllity 
Competency System (ECS) to assist JTPA programs throughout the state in 
implementing comprehensive, competency-based programs. In order to insure that 
the competencies included in the ECS reflected a statewide consensus of the PIC 
approved competencies, a work maturity and pre-employment competency survey of 
all Private Industry Councils in California was conducted by CASAS in the Fall of 
1986 and the Spring of 1987. The competencies kJentified by the majority of PIC's as 
high priority and essential to be included in a competency system, form the basis of 
the Employabllity Competency System. The following components of the system, 
including assessment, are linked to the competencies identified in the statewide 
competency survey of PIC approved pr3-employnientAfvork maturity competencies: 

Employabllity Competency System Appraisal for initial identification of 
basic reading and math functional skill levels. 

Pre-employment Survey Achievement Tests : 

** at three levels (A, B, and C) for monitoring progress in reading , 

** at two levels (A and B) for monitoring progress in math, 

at three levels (A, B, and C) for monitoring progress in 

listening comprehension. 

Pre-employment Certification Tests for two levels (B and C) in reading 
and math for determining level or program completion. 

ECS Assessment Checklist for Educational/Employment 
Development Plans (E/EDPs). (being field tested) 

Pre-employment and Work Maturity Performance Checklists 

which include: Interview Checklist, Work Maturity Skills Checklist, Job 
Application Checklist, Resume Checklist, Cover Letter Checklist, Job Search 
Checklist, and Job Experience/Training Checklist, (being field tested) 

Pre-employment Curriculum Index linked to the statewide PIC approved 
competencies and a model cunriculum which identifies the k>asic skills needed 
to demonstrate pre-employment and wortc maturity competencies across a 
continuum of program levels, (under development) 
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APPENDIX B 



The following counties, agencies and/or Service Delivery Areas (SDA's) returned 
ECS Appraisal answer sheets and are included in the data base used for this report. 
See Figure 18 for the SDA participation percentages. 

Alhambra 

City of Compton - Manpower Department 

El Monte Mid-Valley Manpower Consortium 

Fresno County 

Garden Grove JTPA 

Golden Sien'a Job Training Agency 

Imperial County 

Los Angeles County 

Merced County 

Muroc Joint UHS 

Oceanside 

Ontario Emporium Training Agency 
Riverside County JTPD 
Santa Barbara County 

Santa Ana - County of Orange CSA/Summer Youth Program 

San Diego RETC Program 

Stockton Private Industry Council 

Torrance - Switzer Center - First Step Program 

Ventura County Office of Education 

West Covina 

Westminster 



SERVICE DELIVERY AREA BY PERCENT 




N» 10.003 



APPENDIX C 



The following Service Deliver Areas (SDAs) had agencies that ordered and used the 
ECS Appraisal Program Tests during 1987: 

Alameda County Training & Education Board 
Butte County 

Carson, Lomita, Tendance Consortium 

Contra Costa County 

Foothill Emporium & Training Consortium 

Fresno City/County Consortium 

Golden Sierra Consortium 

Kem/lnyo/Mono Consortium 

Long Beach City 

Los Angeles County 

Marin County 

Merced County 

Monterey County 

Mother Lode Consortium 

Napa County 

North Santa Clara Valley Job Training Consortium (NOVA) 
Oakland City 

Orange County Consortium 
Richmond City 
Riverside County 

Sacramento City/County Consortium 
San Benito County 
San Bernardino City 
San Bernardino County 

San Diego Regional Education & Training Consortium 

San Francisco City & County 

San Joaquin County 

San Luis Obispo County 

San Mateo County 

Santa Ana City 

Santa Barbara County 

Santa Clara County 

Santa Cruz 

South East Los Angeles Consortium (SELACO) 

Solano County 

Sonoma County 

Tulare County 

Ventura County 

Verdugo County 
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